Goto

Collaborating Authors

 black box problem


In defence of post-hoc explanations in medical AI

Hatherley, Joshua, Munch, Lauritz, Bjerring, Jens Christian

arXiv.org Artificial Intelligence

Since the early days of the Explainable AI movement, post-hoc explanations have been praised for their potential to improve user understanding, promote trust, and reduce patient safety risks in black box medical AI systems. Recently, however, critics have argued that the benefits of post-hoc explanations are greatly exaggerated since they merely approximate, rather than replicate, the actual reasoning processes that black box systems take to arrive at their outputs. In this article, we aim to defend the value of post-hoc explanations against this recent critique. We argue that even if post-hoc explanations do not replicate the exact reasoning processes of black box systems, they can still improve users' functional understanding of black box systems, increase the accuracy of clinician-AI teams, and assist clinicians in justifying their AI-informed decisions. While post-hoc explanations are not a "silver bullet" solution to the black box problem in medical AI, we conclude that they remain a useful strategy for addressing the black box problem in medical AI.


Federated learning, ethics, and the double black box problem in medical AI

Hatherley, Joshua, Søgaard, Anders, Ballantyne, Angela, Pauwels, Ruben

arXiv.org Artificial Intelligence

Federated learning (FL) is a machine learning approach that allows multiple devices or institutions to collaboratively train a model without sharing their local data with a third-party. FL is considered a promising way to address patient privacy concerns in medical artificial intelligence. The ethical risks of medical FL systems themselves, however, have thus far been underexamined. This paper aims to address this gap. We argue that medical FL presents a new variety of opacity -- federation opacity -- that, in turn, generates a distinctive double black box problem in healthcare AI. We highlight several instances in which the anticipated benefits of medical FL may be exaggerated, and conclude by highlighting key challenges that must be overcome to make FL ethically feasible in medicine.


A white box solution to the black box problem of AI

Kalmykov, V. L., Kalmykov, L. V.

arXiv.org Artificial Intelligence

Artificial intelligence based on neural networks has made significant progress. However, there are concerns about the reliability and security of this approach due to its lack of transparency. This is the black box problem of AI. Here we show how this problem can be solved using symbolic AI, which has a transparent white box nature. The widespread use of symbolic AI is hindered by the opacity of mathematical models and natural language terms, the lack of a unified ontology, and the combinatorial explosion of search options. To solve the AI black box problem and to implement general-purpose symbolic AI, we propose to use deterministic logic cellular automata with rules based on first principles of the general theory of the relevant domain. In this case, the general theory of the relevant domain plays the role of a knowledge base for the cellular automaton inference. A cellular automaton implements automatic parallel logical inference at three levels of organization of a complex system. Our verification of several ecological hypotheses provides a successful precedent for the implementation of white-box AI. Finally, we discuss a program for creating a general-purpose symbolic AI capable of processing knowledge and ensuring the reliability and safety of automated decisions.


AI's Mystery and Its Demystification

#artificialintelligence

The term'Artificial Intelligence' was originally coined in the year 1955 by John McCarthy, who is known as the Father of Artificial Intelligence. Artificial Intelligence is abbreviated as AI. Although AI was coined in the mid 1950's, many people assume that it is a recent development. Work on AI started way back and continues to be one of the top researched areas even today. AI is predicted to even perform surgeries on its own by the year 2048. So, what exactly is AI? Artificial Intelligence is the effort to simulate human intelligence by the use of machines. Looking at the history of computers and its associated technologies, the initial concept was to make machines that perform a specified work so as to enhance accuracy, increase efficiency and cut down on human-prone errors.


Does AI Improve Human Judgment?

#artificialintelligence

Decision-making has mostly revolved around learning from mistakes and making gradual, steady improvements. For several centuries, evolutionary experience has served humans well when it comes to decision-making. So, it is safe to say that most decisions human beings make are based on trial and error. Additionally, humans rely heavily on data to make key decisions. Larger the amount of high-integrity data available, the more balanced and rational their decisions will be.


Does AI Improve Human Judgment?

#artificialintelligence

Decision-making has mostly revolved around learning from mistakes and making gradual, steady improvements. For several centuries, evolutionary experience has served humans well when it comes to decision-making. So, it is safe to say that most decisions human beings make are based on trial and error. Additionally, humans rely heavily on data to make key decisions. Larger the amount of high-integrity data available, the more balanced and rational their decisions will be.


Will AI Make You Faster/Smarter/Stronger ... Or Just Replace You?

#artificialintelligence

I guess there's always a third option, the one Elon Musk worries about. That is, of course, Skynet. But AI is already here, even if it's not self-aware. The valid question is: what will AI do for us? Will it replace us, as we've seen automation do in factories?


Artificial Intelligence: The Next Generation Anti-Corruption Technology

#artificialintelligence

Artificial Intelligence (AI) can be a useful weapon in the fight against corruption. Its capacity to handle huge data is unrivaled, as is its ability to spot abnormalities or trends, such as in financial transaction data. Some of the ways AI is used in society have skeptics, who fear a society that is more monitored, putting privacy and individual freedom in danger. Let's get into the topic in more detail. Artificial intelligence (AI) refers to technologies that allow machines to simulate human intelligence in order to tackle complicated issues.


Unmasking the Black Box Problem of Machine Learning - InformationWeek

#artificialintelligence

Financial and banking services company Standard Chartered turned to a model intelligence platform to get a clearer picture of how its algorithms make decisions on customer data. How machine learning comes to conclusions and produces results can be a bit mysterious, even to the teams that develop the algorithms that drive them -- the so-called black box problem. Standard Chartered chose Truera to help it lift away some of the obscurity and potential biases that might affect results from its ML models. "Data scientists don't directly build the models," says Will Uppington, CEO and co-founder of Truera. "The machine learning algorithm is the direct builder of the model."


Unmasking the Black Box Problem of Machine Learning - InformationWeek

#artificialintelligence

Financial and banking services company Standard Chartered turned to a model intelligence platform to get a clearer picture of how its algorithms make decisions on customer data. How machine learning comes to conclusions and produces results can be a bit mysterious, even to the teams that develop the algorithms that drive them -- the so-called black box problem. Standard Chartered chose Truera to help it lift away some of the obscurity and potential biases that might affect results from its ML models. "Data scientists don't directly build the models," says Will Uppington, CEO and co-founder of Truera. "The machine learning algorithm is the direct builder of the model."